Image Capturing Apparatus and Recording Method

ABSTRACT

When setting of marking on recording data is performed, the marking is automatically performed at a suitable position corresponding to a status during image capturing. While recording of captured images is being performed, a status of a zoom operation, a moving pattern of a camera, and a feature value of video/audio signals are monitored, and a time point at which a predetermined change has occurred regarding this monitored state is detected. A predetermined data position based on this detected time point is then set as a marking position. Setting information of the marking position is stored in a structure of management information, and is stored in a storage section together with the recording data.

TECHNICAL FIELD

The present invention relates to image capturing apparatuses capable ofrecording captured images as moving image information and to recordingmethods for such image capturing apparatus.

BACKGROUND ART

Efficiently managing moving image information recorded on recordingmedia is desired for image capturing apparatuses, such as video cameras,capable of recording captured images on the recording media as themoving image information. Accordingly, for one thing, it has beencarried out that particular meaningful positions on data of the movingimage information recorded on the recording media is set as sectioningpositions or dividing points. Since setting of such sectioning positionscan be considered to be the same as attaching marks at the datapositions on the moving image information, the setting of the sectioningpositions is referred to as marking hereinafter and the sectioningpositions are referred to as marking positions.

The marking positions set in such a manner can be utilized in a searchat the time of playback of the moving image information. That is, it ispossible to access and start playing the moving image informationrecorded on, for example, a recording medium using a given markingposition as a start point.

Meanwhile, conventional image capturing apparatuses are generallyconfigured to perform marking at a data position corresponding to astart timing of recording (image recording) of captured images on arecording media (Japanese Unexamined Patent Application Publication No.2004-72728).

However, in the configuration of setting only a data positioncorresponding to the start of recording as the marking position in theabove-described manner, marking positions are not set between a startand an end of the recording. For this reason, even in a circumstancewhere the marking is desired during the recording, the marking cannot beperformed for this circumstance. For example, when the duration of asingle recording session becomes long, the recorded content is notmonotonous but some changes are highly likely to occur therein. If themarking is not performed in response to such changes during therecording, a playback function becomes poor.

Accordingly, for example, a configuration for allowing marking positionsto be set manually, for example, through key operations during imagecapturing is also possible. However, when it is taken into considerationthat a cameraperson sets marking positions during image capturing, aninconvenience that the cameraperson cannot concentrate on the imagecapturing may be caused. In this case, as the duration of one recordingsession becomes longer, determination of setting of marking positionsbecomes more difficult for the cameraperson.

Additionally, in recent years, it has been possible to perform editionof moving image information recorded on recording media using imagecapturing apparatuses and personal computers. The utilization of suchedition allows edition to set marking positions at mid-course positionson moving image information recorded during one recording session to beperformed. However, such a post-recording edition imposes a troublesomework, which is to perform marking while confirming the content of themoving image information, on users.

Accordingly, the present invention intends to allow marking to beperformed while one recording session is underway and to allow, byfurther advancing this, setting of marking positions to be performed asefficiently as possible. To this end, the present invention allowsmarking to be automatically performed at appropriate positions accordingto a status during image capturing without depending on, for example,operations or the like for positively setting marking positionsaccording to users' intentions.

DISCLOSURE OF INVENTION

Accordingly, the present invention is configured as an image capturingapparatus in the following manner in view of the above-describedproblems.

More specifically, an image capturing apparatus is configured to includecaptured image acquiring means for acquiring image information throughimage capturing, recording means for recording, as recordinginformation, at least the image information acquired by the capturedimage acquiring means as moving image information, apparatus statedetecting means for detecting that the image capturing apparatus is in apredetermined state while image capturing for acquiring the moving imageinformation to be recorded by the recording means is being performed,and sectioning position setting means for setting a sectioning positionon the moving image information on the basis of timing at which theapparatus state detecting means has detected the predetermined state.

In the above-described configuration, a predetermined state regarding animage capturing apparatus is detected while image capturing foracquiring moving image information to be recorded on, for example, arecording medium (a storage medium) is being performed. In response ofthe detection of this predetermined state, a sectioning position (amarking position) on the moving image information is set (marked). Thismeans that marking is appropriately performed according to apredetermined state of an image capturing apparatus between a start (astart of image recording) and an end of an operation for recording thecaptured images on the recording medium as the moving image information.

On that basis, in this embodiment, a sectioning position is set on thebasis of a timing at which the apparatus state detecting means hasdetected a predetermined state of the image capturing apparatus. Thepresent invention assumes that a time when a specific state of the imagecapturing apparatus occurs often matches a time when an operation isperformed on the image capturing apparatus according to an editorialintention of a cameraperson performing the image capturing or a timewhen a change has occurred in the content of the captured images. On thebasis of this, a result that the sectioning position is set at anappropriate position with reference to the content of the moving imageinformation, such as a sectioning position automatically set on movingimage information by a configuration of the present inventioncorresponds to a time when some sort of change occurs in a stream ofimage contents of the moving image information, can be yielded.

In this manner, the present invention can sets sectioning positions atappropriate positions according to the content of images in the courseof moving image information to be recorded between a start and an end ofrecording. This allows setting (marking) of the sectioning positions onthe moving image information to be performed efficiently, and leads toan improvement of a user's convenience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an example of a configuration of an imagecapturing apparatus supporting recording as an embodiment of the presentinvention.

FIG. 2 is a flowchart showing an example of a procedure for a markingprocess in response to a zoom operation.

FIG. 3 is a diagram showing an example of operations, including amarking process in response to a zoom operation, of a system of an imagecapturing apparatus at the time of recording.

FIG. 4 is a flowchart showing an example of a procedure for a markingprocess according to a detection result of movement of an imagecapturing apparatus itself.

FIG. 5 is a diagram showing an example of operations, including amarking process according to a detection result of movement of an imagecapturing apparatus itself, of a system of an image capturing apparatusat the time of recording.

FIG. 6 is a flowchart showing an example of a procedure for a markingprocess according to a distance to a subject.

FIG. 7 is a diagram showing an example of operations, including amarking process according to a distance to a subject, of a system of animage capturing apparatus at the time of recording.

FIG. 8 is a flowchart showing an example of a procedure of a markingprocess according to a skin color detection result (a feature value of avideo signal).

FIG. 9 is a flowchart showing an example of a procedure for detecting askin color occupancy ratio.

FIG. 10A is a diagram for illustrating a concept and a method of skincolor detection.

FIG. 10B is a diagram showing a concept and a method of skin colordetection.

FIG. 11 is a diagram showing an example of operations, including amarking process according to a skin color detection result, of a systemof an image capturing apparatus at the time of recording.

FIG. 12 is a flowchart showing an example of a procedure of a markingprocess in response to a change in position of an image capturingapparatus.

FIG. 13 is a diagram showing an example of operations, including amarking process in response to a change in position of an imagecapturing apparatus, of a system of an image capturing apparatus at thetime of recording.

FIG. 14 is diagram schematically showing an example of a recordedcontent of recording data including results of setting markingpositions.

FIG. 15A is a diagram showing an example of a manner of displaying arecorded content according to the recorded content of FIG. 14.

FIG. 15B is a diagram showing an example of a manner of displaying arecorded content according to the recorded content of FIG. 14.

FIG. 15C is a diagram showing an example of a manner of displaying arecorded content according to the recorded content of FIG. 14.

FIG. 16 is a diagram showing an example of a configuration of an imagecapturing apparatus of an embodiment supporting playback.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 selectively shows a configuration supporting recording ofcaptured images as an image capturing apparatus that is the best modefor carrying out the invention (an embodiment). In this drawing, solidarrows indicate a flow of information, such as video signals, audiosignals, or management information, that serves as user data to bewritten and stored on a recording medium (a storage medium) in a storagesection. Empty arrows indicate a flow of information for various kindsof control.

An image pickup section 11 is constituted by, for example, an opticalsystem (an optical system for use in image capturing), an image pickupdevice, an image signal processor before compression and encoding, andso forth. The optical system is constituted by a lens unit composed oflenses for image capturing, a lens stop, and so forth. The lens unitincludes a focus lens for adjusting the focus and a zoom lens foradjusting the zoom (an angle of view). The image pickup section 11includes a focus lens driving mechanism constituted by a gear and amotor for moving the focus lens along an optical axis direction in orderto adjust the focus. In addition, the image pickup section 11 includes azoom lens driving mechanism constituted by a gear and a motor for movingthe zoom lens along the optical axis direction in order to adjust thezoom.

In the image pickup section 11, the captured light obtained by theoptical system is imaged on a photoelectric converter unit included inthe image pickup device and is converted into electric signals. A CCD(Charged Coupled Device) and a CMOS can be cited as the photoelectricconverter.

The image pickup section 11 converts the electric signals output fromthe image pickup device into digital signal after performing, forexample, CDS (Correlated Double Sampling) processing, AGC (AutomaticGain Control) processing, and so forth on the electric signals. Theimage pickup section 11 performs predetermined image signal processingaccording to digital signal processing on the digital signal. As thisimage signal processing, necessary digital signal processing including,for example, acquisition of digital video signals corresponding to thecaptured images from the digital signals, is executed.

Additionally, as the signal processing, the image pickup section 11executes necessary signal processing for generation of necessary controlparameters utilized by a camera controlling section 18 for cameracontrol. Here, AF (Auto Focus) control, exposure control, white balanceprocessing, motion blurring correction, and so forth can be cited as thecamera control to be executed by the camera controlling section 18.

For example, in the case of the AF control, the image pickup section 11executes the following signal processing. The AF control is to controlthe camera so that a focused state is automatically obtained not througha manual operation of a focus lens. To achieve the AF control, the imagepickup section 11 is configured to acquire information of a distance toa subject to be focused (herein, referred to a subject distance) orsubject distance information that is information equivalent to thisdistance. For example, if a ranging method for actually measuring adistance to a subject is adopted as the AF control method, informationof the subject distance can be directly obtained. Additionally, in thecase of a so-called contrast method that is based on detection ofcontrast of captured video signals, an evaluation value corresponding tothe contrast is obtained as information equivalent to the subjectdistance information. The image pickup section 11 then outputs suchsubject distance information to the camera controlling section 18. Thecamera controlling section 18 converts the subject distance informationto an amount of the movement of the focus lens, and controls the focuslens driving mechanism of the image pickup section 11 to perform controlof moving the focus lens. By means of such control, a state in which thesubject is focused on can be obtained.

In addition, a movement detecting section 21 is included for the motionblurring correction in this embodiment. The movement detecting section21 of this case includes, for example, a gyro sensor. By fixing themovement detecting section 21 having such a configuration to the imagecapturing apparatus, it is possible to detect the direction of movementand an amount of the movement (a distance of the movement) of the imagecapturing apparatus itself. Thus, in a state where a user carries animage capturing apparatus and performs image capturing, a movement stateof the image capturing apparatus due to the motion blurring at that timeis detected as information of the above-described movement direction andmovement amount.

The camera controlling section 18 receives detection information of, forexample, the above-described movement detecting section 21 and performspredetermined processing, thereby obtaining information of an amount ofmovement of the image capturing apparatus as each component in thehorizontal direction/the vertical direction. On the basis of thisinformation, the camera controlling section 18 executes processing ofthe digital video signals obtained based on the captured images so thatthe movement in the horizontal/vertical directions of the capturedimages due to the motion blurring is canceled.

Meanwhile, regarding the motion blurring correction, a configuration fordetecting movement of an image resulting from the motion blurring byanalyzing a video signal of a captured image and for realizing thecorrection by video signal processing such as shifting of pixels in thehorizontal/vertical directions so that this movement of the image iscancelled is also known. When the image capturing apparatus of thisembodiment adopts such a configuration for motion blurring correction, afunction as the movement detecting section 21 corresponds to, forexample, processing for detecting movement of an image for the motionblurring correction of the image pickup section 11.

Additionally, a position detecting section 24 is provided in the imagecapturing apparatus of this embodiment.

The position detecting section 24 is constituted by a device, such as,for example, a GPS (Global Positioning System), and detects the presentposition based on information of the latitude and the longitude, forexample. In the case of this embodiment, the present positioninformation detected by the position detecting section 24 is input tothe camera controlling section 18. The present position informationacquired in this manner may be utilized in any way including, forexample, using the present position information as additionalinformation of recording data. However, in this embodiment, the presentposition information can be used in marking (automatic setting ofsectioning positions on recording data) in a manner described later.

In addition, the position detecting section 24 may adopt a configurationcapable of acquiring present position information other than thatincluding a GPS.

The digital video signal finally obtained in the image pickup section 11is input to a video compression processing section 12. The videocompression section 12 performs compression and encoding on the inputvideo signal according to a predetermined method. For example, under thepresent conditions, a compression encoding method for moving imageinformation such as MPEG4 can be adopted.

The video signal compressed and encoded by the video compressionprocessing section 12 is output to a multiplexing processing section 15.

In addition, a sound pickup section 13 is configured to include apredetermined number of microphones and audio signal processor forperforming predetermined signal processing, before compression encoding,on audio signals obtained by capturing sound by the microphones. In thiscase, the audio signals are converted from analog to a digital format inthe sound pickup section 13 at a predetermined signal processing stage.

The digital audio signal output from the sound pickup section 13 isoutput to an audio compression processing section 14.

The audio compression processing section 14 executes audio compressionand encoding processing according to the image compression encodingmethod of the video compression processing section 12, and outputs theaudio signal to the multiplexing processing section 15. Here, the videocompression processing section 12 and the audio compression processingsection 14 execute compression and encoding processing so that so-calledlip-sync is maintained, and output the compressed and encoded videosignal (compressed video data) and audio signal (compressed audio data),respectively.

The multiplexing processing section 15 receives the compressed videodata and the compressed audio data output from the video compressionprocessing section 12 and the audio compression processing section 14,and performs multiplexing of the received compressed video data andcompressed audio data according to a data format compatible with arecording format of a storage section 17. The multiplexing processingsection 15 transfers this multiplexed compressed video data andcompressed audio data (compressed video/audio data) to a butteringsection 16 to temporary buffer the data therein.

The camera controlling section 18 is a part collectively demonstratingpredetermined control functions, which are represented by the AFcontrol, the zoom control, and the motion blurring correction describedbefore, for parts constituting a camera unit in the image pickup section11. In addition, information, such as predetermined control resultsobtained in this camera controlling section 18, is received by amanagement information controlling section 20 and is used in generationof management information.

A video controlling section 23 performs various kinds of control so thatthe signal processing operations are appropriately executed in the videocompression processing section 12. For example, the video controllingsection 23 receives data from the video compression processing section12, and executes various kinds of detection processing, such asdetection of movement used for compression processing, and arithmeticprocessing, and performs necessary control of signal processing of thevideo compression processing section 12 according to the detectionresult and the arithmetic processing result. In addition, information ofdetection results and control results regarding predetermined itemsobtained in the video controlling section 23 is received by themanagement information controlling section 20 and is used in generationof management information.

An audio controlling section 19 executes various kinds of controlregarding audio signal processing operations in this case. Accordingly,the audio controlling section 19 controls the signal processingoperations of the audio compression processing section 14 on the basisof predetermined information output according to a signal processingresult of the sound pickup section 13, for example. In addition,information of predetermined control results obtained in this audiocontrolling section 19 is also received by the management informationcontrolling section 20 and is used in generation of managementinformation.

A user interface controlling section 22 is a part for receivingoperation information obtained in response to predetermined operationsperformed on various operation buttons equipped with the image capturingapparatus, and for processing the operation information so as to allowthe image capturing apparatus to execute appropriate works correspondingto the operations. The user interface controlling section 22 transmitsthe operation information obtained in response to predeterminedoperations to necessary parts in each functional part shown in thedrawing here. The management information controlling section 20 utilizesthe operation information transmit from the user interface controllingsection 22 in generation of predetermined management information.

The management information controlling section 20 generates managementinformation. This management information is data to be recorded on thestorage section 17 (a recording medium) together with the recording dataaccording to a predetermined structure. The management information isformed to include information of a predetermined content for managingthe video/audio data that is generated by multiplexing each data of thevideo signals obtained in the image pickup section 11 and the audiosignals obtained in the sound pickup section 13. In addition to theabove-described video/audio data to be managed, this managementinformation has an information structure and the information contentcompliant with a format of a recording medium actually supported by thestorage section 17.

In addition, as described above, the management information controllingsection 20 utilizes information, such as predetermined controlinformation and detection results output from, for example, the cameracontrolling section 18, the audio controlling section 19, and the videocompression processing section 12, in generation of managementinformation.

The above-described management information in this embodiment has apredetermined structure that allow sectioning positions to be setregarding the video/audio data obtained by image capturing/soundcapturing. This sectioning position can be defined as, for example, aposition (time position) corresponding to a time given to thevideo/audio data by information such as so-called timestamp.

Additionally, the image capturing apparatus of this embodiment sets thissectioning position at the least in the following manner. Firstly, theimage capturing apparatus sets a recording start position of thevideo/audio data as a sectioning position in response to a user'srecording start operation. In addition, the sectioning position can alsobe set at a middle of the video/audio data from the start to the end ofthe recording. In this embodiment, this middle sectioning position canbe automatically set by a marking process to be described later.Meanwhile, the recording in this embodiment means to record data(compressed video/audio data) of the video signal and the audio signal,obtained by capturing images and sound in the image pickup section 11and the sound pickup section 13, on the storage section 17. In addition,the compressed video/audio data to be recorded on the recording mediumis also referred to as recording data. The recording data in thisembodiment is at least data of the video signal. However, in practice,the recording data may include an audio signal whose playback time issynchronized with that of this video signal.

The management information generated by the management informationcontrolling section 20 is transferred to the buffering section 16 andtemporarily buffered therein in order to be recorded on a recordingmedia.

The compressed video/audio data transferred from the multiplexingprocessing section 15 and the above-described management information arebuffered in the buffering section 16.

The buffering section 16 transfers a predetermined amount of thebuffered compressed video/audio data to the storage section 17 at apredetermined timing, and performs writing. Similarly, the bufferingsection 17 transfers the buffered management information to the storagesection 17 at a predetermined timing, and performs writing. Bysequentially performing recording of data in such a manner, the contentof the compressed video/audio data obtained by image capturing/soundcapturing is recorded together with the management information.

As a specific configuration of the storage section 17, a predeterminedrecording medium (a storage medium) and a drive capable of recording(and playing) data compatible with this recording medium are included.In this case, the recording medium may be, for example, removable or maybe implemented with a drive in a fixed fashion, such as, for example, anHDD (hard disk).

In addition, for example, when the storage section 17 supports DVDs(Digital Versatile Disc), a so-called DVD format such as DVD-Video orDVD-VR (Video Recording) is adopted as a recording format of the storagesection 17.

The configuration shown in this drawing shows functions regarding imagerecording (and sound recording) of an image apparatus through a blockconfiguration, and differs from the actual hardware configuration. Inpractice, the image capturing apparatus is constituted by, for example,a part that performs image capturing with image pickup devices, a partthat performs sound capturing with microphones, a processor thatconverts a video signal and an audio signal obtained by these parts intodigital signals and that executes predetermined digital signalprocessing, encoding processing, and multiplexing processing, a drivethat supports a predetermined recording medium, a CPU, a ROM, a RAM, anda microcomputer that executes various kinds of necessary controlprocessing.

In this embodiment, with the above-described configuration, a markingoperation for setting a sectioning position on recording data, to berecorded on a recording media through image recording, is automaticallyexecuted while the image recording is being performed. In addition, thisembodiment provides a mechanism for marking a noteworthy meaningfulposition on recording data according to a status during the imagerecording, so that users' convenience is considered.

Now, the noteworthy meaningful position on recording data is considered.For example, a timing at which some sort of change occurred at acaptured subject, or a timing at which a cameraperson pays attention toa subject again and takes an action such as some sort of operation asthe cameraperson's intention can be considered to be noteworthy andmeaningful in the image contents of the recording data. That is, whenthe cameraperson pays attention to a subject or when a subject changesinto a noteworthy state, there are changes in an operation of an imagecapturing apparatus and it can be considered that there are oftenchanges in the content of images and audio to be recorded.

Accordingly, in this embodiment, a noteworthy timing on theabove-described recording data is acquired as results of detecting astate of an image capturing apparatus, an operation of the imagecapturing apparatus, or the like on the basis of such facts. Accordingto this detected timing, marking is performed, that is, a sectioningposition is set on the recording data.

In this embodiment, for example, the following configurations arepossible to perform marking in response to the noteworthy meaningfultiming in the course of the recording data.

One is a configuration for allowing marking to be performed in responseto a timing at which a user has performed an operation to adjust thezoom during recording.

It is assumed that the image capturing apparatus of this embodimentincludes a zoom function. The zoom function is a function for enlargingor reducing captured images by, for example, adjusting an angle of viewof the lens optical system in the image pickup section 11. The angle ofview is optically adjusted variably by moving the zoom lens in the lensoptical system of the image pickup section 11 along the optical axisdirection. In addition, a zoom function for enlarging or reducing imagesnot by optical means but by image signal processing is also known. Auser who is the cameraperson performs a predetermined operation (a zoomoperation) for zoom adjustment on the image pickup section 11, wherebysuch zoom adjustment works.

Now, circumstances where the cameraperson performs a zoom operation willbe considered. The circumstance is when the cameraperson has some sortof editorial intentions, such as, for example, when the camerapersonzooms in (narrows the angle of view) to emphasize a subject and makesthe subject look larger in the composition, or when the camerapersonzooms out (broadens the angle of view) from a state where the subject isclosed up so that a whose scenery is captured. That is, it can beconsidered that, as the content of the recording data, the circumstancecorresponds to some sort of noteworthy and meaningful changing points.

On that basis, the following conditions can be considered as timings ofmarking in response to a zoom operation.

a. A time point at which a zoom-in operation is started is set as amarking timing.

b. A time point at which a zoom-in operation is terminated is set as amarking timing.

c. A time point at which a zoom-out operation is started is set as amarking timing.

d. A time point at which a zoom-out operation is terminated is set as amarking timing.

e. A time point at which a predetermined time has passed since a timepoint of the start of a zoom-in operation is set as a marking timing.

f. A time point at which a predetermined time has passed since a timepoint of the termination of a zoom-in operation is set as a markingtiming.

g. A time point at which a predetermined time has passed since a timepoint of the start of a zoom-out operation is set as a marking timing.

h. A time point at which a predetermined time has passed since a timepoint of the termination of a zoom-out operation is set as a markingtiming.

i. A time point that is a predetermined time before a time point of thestart of a zoom-in operation is set as a marking timing.

j. A time point that is a predetermined time before a time point of thetermination of a zoom-in operation is set as a marking timing.

k. A time point that is a predetermined time before a time point of thestart of a zoom-out operation is set as a marking timing.

l. A time point that is a predetermined time before a time point of thetermination of a zoom-out operation is set as a marking timing.

For example, the marking is set at a timing at which any one of theseconditions is met.

The above-described conditions a-d employ timings directly correspondingto the start/termination of a zoom-in/zoom-out operation, and can beconstituted by, for example, simple algorithms.

In addition, when timings at which a predetermined time has passed sincea time point of the start/termination of a zoom-in/zoom-out operation isset as marking timings like the conditions e-l, the following advantagesexist. For example, while the zoom lens is moving in response to a zoomoperation, a captured image also changes to enlarge/reduce in responseto this. For this reason, if the timing directly corresponding to thestart/termination of a zoom-in/zoom-out operation is set as the markingtiming, the image changes in response to the zoom operation immediatelybefore or immediately after this marking timing. In addition, thecaptured composition is often shifted at the timing of thestart/termination of the zoom-in/zoom-out operation. That is, asectioning position on the recording data is set at a point where achange in images is great, which make it difficult to see when playbackof the recording data is performed using the marked sectioning positionas a start point. By providing a predetermined time from the operationtiming like the conditions e-l, the marking timing is set for a statewhere images are stable.

In addition, regarding the conditions i-l, the timing that is apredetermined time before a time point of the start/termination of azoom operation is set as a sectioning position. Such marking is possiblewhile the recording is performed in realtime. In the case of recordingdigital recording data in a manner of this embodiment, the recordingdata is temporarily buffered, for example, in the buffering section 16,and then sequentially written on a recording medium at a predeterminedtiming. That is, a certain amount of information of images and audiocaptured before the current time point is not recorded on a recordingmedium but is buffered. In addition, depending on the format of therecording media, the sectioning position of this embodiment is includedin a structure of the management information and is written in an areaon the recording media that is logically different from that for theentity of the recording data (compressed video/audio data). In thiscase, the management information is held in a buffer until thecompletion of the recording, for example, and is written on a recordingmedium at a predetermined timing corresponding to the completion of therecording. For this reason, a past time on the recording data can be set(marked) as a sectioning position by rewriting this managementinformation as long as the management information is buffered.

In addition, it is possible to combine the above-described conditionsa-l and the following marking execution determination criterion. Morespecifically, the execution determination criterion is that this markingis executed if a predetermining time or more has passed from the timingof the lastly executed marking to the timing of this marking that isdecided according to one of the above-described conditions a-l, whereasthis marking is not executed if the predetermined time has not passed.

When such an execution determination criterion is not set, markingpositions are set every time a cameraperson performs a zoom operationduring a short period of time due to a reason that the composition isnot well set, for example. That is, marking positions more than thenecessary number may be set during a short period of time.

Another configuration is that for allowing marking to be performedaccording to a moving state (a movement state) of an image capturingapparatus due to panning or tilting during the recording.

The image capturing apparatus of this embodiment is generally used in amanner that a cameraperson performs image capturing while carrying theimage capturing apparatus and looking through a viewfinder or looking ata display screen. On the basis of this, when a direction change equal toor greater than a predetermined angle is occurred as a moving state ofthe image capturing apparatus, for example, a circumstance, such as thecameraperson has an intention to change a subject or a significantchange has occurred in a scene because an identical subject movesgreatly, can be expected. As in the case of the above-described zoomoperation, it can be considered as a noteworthy and meaningful timing asrecording data.

In this embodiment, the moving state (movement state) of the imagecapturing apparatus can be grasped on the basis of detection informationfor motion blurring correction of the movement detecting section 21. Themovement detecting section 21 includes a geomagnetic sensor, a gyro, andan acceleration sensor, thereby knowing a rotation of the imagecapturing apparatus (a direction), an amount of movement of the imagecapturing apparatus, and a moving direction in the horizontal direction.In addition, a rotation of an image capturing apparatus in a capturingdirection, an amount of movement of the image capturing apparatus, and amoving direction in the up and down (vertical) direction can be known.

The following conditions can be considered as marking timings of thiscase.

a. A time point at which a rotation of an image capturing apparatus inthe horizontal or vertical direction is started is set as a markingtiming.

b. A time point at which a rotation of an image capturing apparatus inthe horizontal or vertical direction is terminated is set as a markingtiming.

c. A time point at which a liner movement of an image capturingapparatus in the horizontal or vertical direction is started is set as amarking timing.

d. A time point at which a liner movement of an image capturingapparatus in the horizontal or vertical direction is terminated is setas a marking timing.

e. A time point at which a predetermined time has passed since a timepoint of the start of a rotation of an image capturing apparatus in thehorizontal or vertical direction is set as a marking timing.

f. A time point at which a predetermined time has passed since a timepoint of the termination of a rotation of an image capturing apparatusin the horizontal or vertical direction is set as a marking timing.

g. A time point at which a predetermined time has passed since a timepoint of the start of a linear movement of an image capturing apparatusin the horizontal or vertical direction is set as a marking timing.

h. A time point at which a predetermined time has passed since a timepoint of the termination of a linear movement of an image capturingapparatus in the horizontal or vertical direction is set as a markingtiming.

In this case, the marking is also set at a timing at which any one ofthese conditions is met.

In addition, also in this case, it is possible to combine a markingtiming execution determination criterion similar to that in the case ofzooming (this marking is executed if a predetermined time or more haspassed since the timing of the lastly executed marking) with theabove-described conditions. This can suppress setting of unnecessarymarkings.

Another configuration is that for allowing marking to be performed onthe basis of a change in a subject distance obtained by capturing imagesduring the recording.

The image capturing apparatus of this embodiment includes an auto focusfunction. The auto focus function is a function for controlling theimage capturing apparatus so that a subject focused state isautomatically obtained not by manual operation of a focus lens by auser. The auto focus control generally determines a subject distancethat is a distance to a subject or information equivalent to this (anevaluation value (contrast (difference in brightness levels)information) obtained from a video signal on the basis of the contrastmethod, for example), drives the focus lens to move according to thissubject distance, thereby obtaining the focused state. Here, a fact thatthe subject distance shows a certain characteristic change means that itis highly likely that the user performing the image capturingintentionally applies some sort of change to the captured content. Thatis, when the subject distance shows a characteristic change, the contentof the recording data is also considered to be noteworthy andmeaningful.

For example, the followings can be cited as conditions of markingtimings of this case.

a. A time point at which a continuous decrease or a continuous increaseof a subject distance stops is set as a marking timing.

b. A time point at which a decreasing or increasing trend of a change ofa subject distance starts is terminated is set as a marking timing.

c. A time point at which a predetermined time has passed since a timepoint of the stop of a continuous decrease or a continuous increase of asubject distance is set as a marking timing.

d. A time point at which a predetermined time before a time point of thestart of a decreasing or increasing trend of a change of a subjectdistance is set as a marking timing.

e. A time point at which a continuous decreasing or a continuousincreasing subject distance falls within a predetermined range is set asa marking timing.

f. A time point that is a predetermined time after or before a timepoint at which a continuous decreasing or a continuous increasingsubject distance falls within a predetermined range is set as a markingtiming.

In this case, it is also possible to include a marking timing executiondetermination criterion (this marking is not executed unless apredetermined time or more has passed since the timing of the lastexecuted marking) in the above-described conditions.

Another configuration is that for detecting a change regarding apredetermined feature of an input signal (a video signal or an audiosignal) obtained by image capturing or sound capturing, and for performmarking according to this detection result. The timing at which an inputsignal changes can be also considered as a noteworthy and meaningfultiming as a content of the recording data. Features of an input signalthat can be utilized in marking will be illustrated below.

A. Luminance of image: The luminance can be detected on the basis of aluminance signal component of a video signal.

B. Color of image: It can be considered that particularly detecting skincolors in a specific area of an image as a feature is effective when ahuman figure is set as a subject.

C. Complexity of content of image: The complexity can be determined bydigitizing a correlation between a coefficient that decides acompression ratio used when a video signal is compressed and encoded bythe video compression processing section 12 and an amount of compressedinformation compressed using the coefficient.

D. Volume of captured audio: The volume can be detected on the basis ofa level of an audio signal.

E. Localization of sound source regarding captured audio: When the imagecapturing apparatus of this embodiment supports multichannel soundcapturing, a change in localization of a sound source of the capturedsound can be recognized on the basis of information of an audio signalof each channel obtained by the sound capturing. For example, thelocalization can be obtained as an audio signal processing result of thesound pickup section 13.

F: Peripheral noises: For example, when the sound pickup section 13includes a noise canceller that cancels external noises, a change in thenoises can be detected on the basis of a change in a controlling amountof this noise canceller.

When a predetermined change occurs in any one of the features of aninput signal shown as the above-described A-F, it can be considered thata noteworthy and meaningful change occurs in the content of capturedimages at that time.

In addition, the followings are cited as marking timings of this case.

a. A time point at which it is determined that a value (a feature value)indicating a feature of an input signal continues to increase ordecrease and then becomes constant (or falls within a predeterminedrange of values) is set as a marking timing.

b. A time point at which it is determined that a feature value of aninput signal starts to increase or decrease is set as a marking timing.

c. A time point at which a predetermined time has passed since it isdetermined that a value (a feature value) indicating a feature of aninput signal continues to increase or decrease and then becomes constant(or falls within a predetermined range of values) is set as a markingtiming.

d. A time point at which a predetermined has passed since it isdetermined that a feature value of an input signal starts to increase ordecrease is set as a marking timing.

In this case, for example, the marking is also set at a timing at whichany one of these conditions is met. Limiting the application of theabove-described conditions to a specific area of an image area for oneframe in the case of images and to a specific frequency band in the caseof audio is also possible.

In this case, it is possible to include a marking timing executiondetermination criterion (this marking is not executed unless apredetermined time or more has passed since the timing of the lastlyexecuted marking) in the above-described conditions.

The configuration for performing marking according to the movement stateof the image capturing apparatus itself due to panning or tilting duringthe recording has been described before. A configuration for allowingmarking to be performed on the basis of particularly a position changeaccording to the movement distance as the movement state is alsopossible.

More specifically, for example, when a user carrying an image capturingapparatus moves to a position apart from the position of an originallocation by a predetermined distance, the user is in a location wherethe environment is different from that of the original location. If theimage capturing is performed during this period, the timing at whichsuch a position change occurs can be considered as a noteworthy timingas the recording data.

The image capturing apparatus of this embodiment can grasp the somedegree of distance change by utilizing the position information detectedby the position detecting section 24.

The following conditions can be considered as marking timings of thiscase.

a. A time point at which a distance between a position detected at thetime of the last marking and the present position becomes equal to orgreater than a predetermined value is set as a marking timing.

b. A time point at which a predetermined time has passed since a timepoint at which a distance between a position detected at the time of thelast marking and the present position becomes equal to or greater than apredetermined value is set as a marking timing.

c. A time point that is a predetermined period before a time point atwhich a distance between a position detected at the time of the lastmarking and the present position becomes equal to or greater than apredetermined value is set as a marking timing.

d. A time point at which a distance between a predetermined referenceposition and the present position becomes equal to or greater than apredetermined value is set as a marking timing.

e. A time point at which a predetermined time has passed since a timepoint at which a distance between a predetermined reference position andthe present position becomes equal to or greater than a predeterminedvalue is set as a marking timing.

f. A time point that is a predetermined period before a time point atwhich a distance between a predetermined reference position and thepresent position becomes equal to or greater than a predetermined valueis set as a marking timing.

g. A time point at which a distance between a predetermined referenceposition and the present position becomes equal to or smaller than apredetermined value is set as a marking timing.

h. A time point at which a predetermined time has passed since a timepoint at which a distance between a predetermined reference position andthe present position becomes equal to or smaller than a predeterminedvalue is set as a marking timing.

i. A time point that is a predetermined period before a time point atwhich a distance between a predetermined reference position and thepresent position becomes equal to or smaller than a predetermined valueis set as a marking timing.

In this embodiment, a data position treatable as a noteworthy andmeaningful content in recording data is detected and recognizedutilizing functions included in the image capturing apparatus based onthe state of operation executed by the image capturing apparatus, theoperation result, and the state of the image capturing apparatus in thismanner. On the basis of this detection result or recognition result, thetiming of marking is set. This allows sectioning positions of recordingdata to be set automatically. In addition, the result of setting thesectioning positions specifies a suitably meaningful data positionsaccording to the content of the recording data.

Subsequently, a configuration for allowing the image capturing apparatusof this embodiment to execute marking will be described for some of theabove-described condition setting examples.

Firstly, a configuration for performing marking in response to a zoomoperation will be described using FIG. 2 and FIG. 3. Meanwhile, a casewhere an execution determination criterion that “marking is notperformed unless one minute or more has passed from the last marking tothe set marking timing” is set under a condition that “a time point thatis one second before a time point at which a zoom-in operation isstarted is set as marking timing” is cited here.

A flowchart of FIG. 2 shows a procedure according to an algorithm forexecuting marking in response to a zoom-in operation. The procedure ofthis FIG. 2 can be seen as, for example, processing executed by themanagement information controlling section 20.

Firstly, a start of a zoom-in operation is waited for at STEP S101. Forexample, when an operation for zoom-in is started by a user, the userinterface controlling section 22 outputs, in response to this,information indicating that the zoom-in operation has started to themanagement information controlling section 20. The managementinformation controlling section 20 recognizes that the zoom-in operationhas started by receiving this information, and advances the process toSTEP S102.

At STEP S102, the management information controlling section 20 acquiresthe present time Tpre. In this case, the time may be information of aplayback time corresponding to the recording data. For example, acurrent time based on a starting time point of recording data may beacquired as the preset time Tpre.

At the following STEP S103, whether or not one minute or more has passedsince the final (last) marking is determined. To do this, a time for asectioning position set by the last marking may be compared with thepresent time Tpre acquired at STEP S102 described above.

If a negative determination result that one minute or more has notpassed is obtained at STEP S103, it is determined that the markingshould not be performed. Thus, the processing is terminated withoutperforming the marking. On the other hand, if a positive determinationresult that one minute or more has passed is obtained, the processproceeds to STEP S104. A position on the recording data corresponding toTpre−1 second is set as a sectioning position. That is, the marking isperformed at a position on the recording data for a time point that isone second before the time point of the start of the zoom-in operation.

By performing the marking at one second before the start of the zoom-inoperation in this manner, stable images before the zooming are displayedfirst and then shifted to images showing a subject enlarging in responseto the zoom-in when playback is started from this marking position, forexample. By allowing stable images to be displayed at first in thismanner, viewability for users is considered.

FIG. 3 is executed by an image capturing apparatus between a start andan termination of recording of one piece of recording data, and is aspecific example of a sequence of processing for recording the recordingdata including processing for marking in response to a zoom operation.The marking processing in this processing sequence employs the conditionand the execution determination criterion shown in FIG. 2.

In addition, description of this drawing is given according to numeralsindicating steps shown with ( ) in the drawing. In this drawing, thesteps are shown in association with user operations/actions, operationsof user interface control, camera control, compression processing,buffer control, management information control, file system control, andthe storage section 17. Regarding each of the above-describedoperations, the user interface controlling section 22 performs the userinterface control and the camera controlling section 18 performs thecamera control. In addition, the video compression processing section12, the audio compression processing section 14, and the multiplexingprocessing section 15 perform the compression processing, and the buffercontrol is performed by the buffering section 16. The managementinformation controlling section 20 performs the file system control.Information serving as a file system is also management informationnecessary for managing the recording data recorded on, for example, arecording medium so that the recording data is appropriately played backin a unit of file. Storage (storing of data) is executed in the storagesection 17. In addition, the processing in this drawing is in accordancewith the course of time. The time advances as the process proceeds fromthe top to the bottom of the drawing. These points are the sameregarding FIG. 5, FIG. 7, and FIG. 9, which will be described later.

STEP 1: A user, whose is a cameraperson, of an image capturing apparatusperforms a predetermined operation to start recording of recording data.

STEP 2: Operation information corresponding to the recording startoperation is transmitted to the user interface controlling section 22.

STEP 3: In response to the user interface controlling section 22receiving the recording start operation, the video compressionprocessing section 12 starts compression encoding processing, performedby the video compression processing section 12, on a video signalobtained by the image pickup section 11.

Although not shown in the drawing, in response to the user interfacecontrolling section 22 receiving the recording start operation, theaudio compression processing section 14 starts compression encodingprocessing on an audio signal obtained by the sound pickup section 13 ata timing synchronized with that of the video data. The multiplexingprocessing section 15 then generates recording data of multiplexedcompressed video data and compressed audio data.

STEP 4: The recording data generated in the above-described manner issequentially transferred to, written in, and buffered (accumulated) inthe buffering section 16 so that the recording data is sectioned in acertain data size based on, for example, an actual buffer size of thebuffering section 16 and the speed of the compression encodingprocessing. At this STEP 4, a section 1 from the top of the compressedand encoded data to a certain data position is transferred to andbuffered in the buffering section 16.

Data obtained by compressing and encoding video/audio signals obtainedby image capturing/sound capturing thereafter belongs to a section 2.

STEP 5: In response to buffering of the compressed and encoded data ofthe section 1, management information for the compressed and encodeddata of the section 1 is created as the management information control.STEP 6: The management information corresponding to the compressed andencoded data of the section 1 created at STEP 5 is transferred to andbuffered in the buffering section 16.

STEP 7: An update of a file system according to the recoding result ofthe section 1 is performed at a timing corresponding to recording of thecompressed and encoded data of the section 1 on a recording medium atSTEP 8, which will be described next. At this time, for example, themanagement information controlling section 20 reads out data of filesystem held in the buffering section 16, and rewrites the contentaccording the record of the data of the section 1.

STEP 8: The compressed and encoded data of the section 1 is read outfrom the buffering section 16 and is transferred to the storage 17, andis written and stored on the recording medium.

STEP 9: If the update of the file system at STEP 7 is completed, thedata of the update-completed file system is written back in and bufferedin the buffering section 16. At this time, if data of the unupdated filesystem exists in the buffering section 16, the data of the unupdatedfile system is overwritten by the data of the updated file system.

STEP 10: A user performs an operation for starting zoom-in. As shown inthe drawing, a timing of starting the zoom-in operation is while thecompressed and encoded data of the section 1 is being recorded on therecording medium and while the video compression processing section 12is executing generation of compressed and encoded data of the section 2that follows the section 1. That is, the start of this zoom-in operationis performed at a timing within a period during which the images of thedata of the section 2 are captured.

STEP 11: Upon detecting the zoom-in start operation of theabove-described STEP 10, the user interface controlling section 22notifies the management information controlling section 20 that thezoom-in start operation is performed. Although not shown in the drawing,upon detecting the zoom-in start operation, the user interfacecontrolling section 22 notifies the management information controllingsection 20 and also sends the information of the zoom-in operation to,for example, a camera controlling section 18. The camera controllingsection 18 controls the zoom mechanism of the image pickup section 11 toexecute control for zoom adjustment.

STEP 12: The management information controlling section 20 acquires atiming of the zoom-in start operation notified at STEP 11 describedabove as, for example, the present time Tpre (STEP S102), and sets atime that is one second before this present time Tpre as a sectioningposition (a marking position) of the recording data (STEP S104). For theconfirmation, this STEP 12 is a timing within a period during which thedata of the section 2 is being compressed and encoded. That is, thesectioning position (the marking position) for the timing of the presenttime Tpre-1 second set at STEP 12 resides within the recording data ofthe section 2.

If the start of the zoom operation shown as STEP 10 is not the first onebut the second or following one and a predetermined time (one minute)has not passed from the marking position set in response to a start ofthe last zoom operation, setting of the marking position at STEP 12 isnot executed according to the procedure of FIG. 2 (STEP S103).

STEP 13: The user stops the zoom-in operation.

STEP 14: The user interface controlling section 22 detects theabove-described stop of the zoom-in operation.

STEP 15: The user interface controlling section 22 notifies themanagement information controlling section 20 of the termination of thezoom-in operation. Although not shown in the drawing, the user interfacecontrolling section 22 also notifies the camera controlling section 18of the termination of the zoom-in operation. In response to thisnotification, the camera controlling section 18 stops the zoom control.

STEP 16: The compressed and encoded data of the section 2 following thesection 1 is buffered in the buffering section 16. Thereafter,processing for generating compressed and encoded data of a section 3 isexecuted.

STEP 17: In response to buffering of the compressed and encoded data ofthe section 2, management information for the compressed and encodeddata of the section 2 is created as the management information control.At this time, the management information is created so that the markingposition (the sectioning position) of the recording data set atpreceding STEP S12 is reflected. As described before, in thisembodiment, the management information has a structure to storeinformation indicating sectioning positions of recording data.

STEP 18: The management information for the compressed and encoded dataof the section 2 created at STEP 17 is transferred to the bufferingsection 16, and is buffered.

STEP 19: At a timing corresponding to recording of the compressed andencoded data of the section 2 on the recording medium at STEP 20, anupdate of the file system according to the recording result of thesection 2 is performed.

STEP 20: The compressed and encoded data of the section 2 is read outfrom the buffering section 16 and is transferred to the storage 17, andis written and stored on the recording medium.

STEP 21: In response to the completion of the update of the file systemat STEP 19, the data of the update-completed file system is written backto the buffering section 16 and is buffered.

STEP 22: An operation for stopping the recording of the recording datais performed.

STEP 23: The user interface controlling section 22 detects thatrecording stop operation of the recording data has been performed. Inresponse to this, for example, the video compression and encodingprocessing and the audio compression and encoding processing arestopped.

STEP 24: In response to the recording stop operation of the recordingdata, the buffering section 16 that performs the buffering controlbuffers the compressed and encoded data of the section 3.

STEP 25: In response to buffering of the compressed and encoded data ofthe section 3, management information for the compressed and encodeddata of the section 3 is created as the management information control.

STEP 26: The management information for the compressed and encoded dataof the section 3 created at STEP 24 is transferred to the bufferingsection 16 and is buffered.

STEP 27: At a timing corresponding to recording of the compressed andencoded data of the section 3 on the recording medium at STEP 25, anupdate of the file system according to the recording result of thesection 3 is performed.

STEP 28: The compressed and encoded data of the section 3 is read outfrom the buffering section 16, is transferred to the storage 17, and iswritten and stored on the recording medium.

STEP 29: In response to the completion of the update of the file systemat STEP 27, the data of the update-completed file system is written backto the buffering section 16 and is buffered. In this case, the data ofthe file system is written and stored on the recording medium at anecessary timing thereafter.

Subsequently, a configuration for performing marking according to amovement state of the image capturing apparatus itself during recordingwill be described.

In this case, a condition that “movement of an image capturing apparatusin the horizontal direction (which may be a rotation or a linearmovement) has once started and a time point at which a predeterminedtime of one second has passed from a time point at which the movementhas stopped is set as a marking timing” is set. In addition, anexecution determination criterion that “marking is not performed unlessone minute or more has passed from the last marking to the set markingtiming” is also included. Furthermore, an execution determinationcriterion that “marking is performed if the recording is continued (hasnot finished) at the time corresponding to the set marking timing,whereas marking is not performed if the recording is not continued (hasfinished)” is additionally included. In this case, since the marking isperformed at a time after the timing of detecting the end of themovement of the image capturing apparatus in the horizontal direction,the recording may be finished at the timing of detecting the end of themovement. That is, the latter execution determination criterion solvesthe processing contradiction that marking processing is executed inspite of that the recording has finished.

A flowchart of FIG. 4 shows an example of a procedure for executingmarking according to a movement state of an image capturing apparatus.

Firstly, a stop of the movement is waited for at STEP S201 aftermovement of an image capturing apparatus itself has started in thehorizontal direction, for example. As described above, thisdetermination is possible on the basis of information detected by themovement detecting section 21. In the drawing, the managementinformation controlling section 20 is configured to receive theinformation of the movement detecting section 21, for example, throughthe camera controlling section 18. Alternatively, this determination isalso possible on the basis of a result of performing detection ofmovement regarding a captured image through image signal processing. Ifa positive determination result is obtained at STEP S201, the processproceeds to STEP 202 and the following steps.

At STEP S202, the present time Tpre is acquired.

At STEP S203, whether or not one minute or more has passed since thefinal (last) marking is determined. If a negative determination resultis obtained, the processing is terminated without performing themarking. On the other hand, if a positive determination result that oneminute or more has passed is obtained, the process proceeds to STEPS204.

At STEP S204, whether or not the recording is continued at a time pointof one second after the present time Tpre is determined. To do this, forexample, the image capturing apparatus may wait for one second from thepresent time Tpre acquired at STEP S202 and then check an operationstate of the image capturing apparatus at this time. If a negativedetermination result is obtained at this STEP S204, the process shown inthis drawing is terminated without performing the marking. This solvesthe processing contradiction that the marking processing is executed inspite of that the recording has finished.

On the other hand, if a positive determination result is obtained atSTEP S204, a position on recording data corresponding to the presenttime Tpre+1 second is set as a sectioning position at STEP S205. Thatis, marking is performed at the position on the recording data for atime point that is one second after the time point of the start of thezoom-in operation. By performing the marking at the timing one secondafter the stop of the movement in this manner, it is possible to playback images from a state where blurring of images due to the movement ofthe image capturing apparatus is removed when the images are played backfrom, for example, this marking position since the playback is performedfrom the time point that is one second after the stop of the movement ofthe image capturing apparatus.

FIG. 5 is executed by an image capturing apparatus between a start and atermination of recording of one piece of recording data, and is aspecific example of a sequence of processing for recording the recordingdata including processing for marking according to movement (in thehorizontal direction) of the image capturing apparatus. This processingsequence will be described according to numerals indicating steps shownwith ( ) in the drawing. The marking processing in this processingsequence employs the condition and the execution determination criteriashown in FIG. 4.

Since STEP 1 to STEP 6 are similar to STEP 1 to STEP 6 of FIG. 3, thedescription is omitted here.

In addition, writing of data of a section 1, an update of a file system,and buffering of STEP 7 to STEP 9 are also similar to those of FIG. 3.

STEP 10: In this drawing, for example, an action involving movement ofan image capturing apparatus, such as movement of a cameraperson (auser) carrying the image capturing apparatus or changing of a directionof the image capturing apparatus, is started as this STEP 10. In thiscase, a timing of the start of the movement of the image capturingapparatus is while compressed and encoded data of a section 1 is beingrecorded on a recording medium and while the video compressionprocessing section 12 is executing generation of compressed and encodeddata of a section 2 that follows the section 1.

STEP 11: In response to the start of the movement of the image capturingapparatus at the above-described STEP 10, the movement detecting section21 outputs information of the movement state (a moving direction and anamount of the movement). The user interface controlling section 22receives such detection information from the movement detecting section21.

STEP 12: The user interface controlling section 22 recognizes the amountof movement and the moving direction regarding vertical/horizontalcomponents from, for example, the detection information of the movementdetecting section 21. On the basis of this recognition result, the userinterface controlling section 22 notifies the management informationcontrolling section 20 that the movement in the horizontal direction hasstarted. Meanwhile, since the marking condition used here is one secondafter a time point of a termination of the movement in the horizontaldirection, setting of marking is not performed at STEP 12.

STEP 13: The user's action for moving the image capturing apparatus inthe horizontal direction is stopped. This terminates the movement of theimage capturing apparatus in the horizontal direction.

STEP 14: Detection information of the movement detecting section 21 inresponse to the termination of the movement of the image capturingapparatus in the horizontal direction is input to the user interfacecontrolling section 22. The user interface controlling section 22recognizes that the movement of the image capturing apparatus in thehorizontal direction has been terminated on the basis of the inputdetection information of the movement detecting section 21, and notifiesthe management information controlling section 20 that the movement ofthe image capturing apparatus in the horizontal direction has beenterminated.

STEP 15: The management information controlling section 20 notified thatthe movement of the image capturing apparatus in the horizontaldirection has been terminated acquires a timing of the termination ofthe movement in the horizontal direction notified at the above-describedSTEP 14 as, for example, the present time Tpre (STEP S202 of FIG. 4).The management information controlling section 20 sets a time that isone second after the this present time Tpre as a sectioning position (amarking position) of the recording data (STEP S205). Meanwhile, thesectioning position (the marking position) set in this manner is alsoset within recording data of a section 2.

Since STEP 16 to STEP 29 are similar to those of FIG. 3, the descriptionis omitted here.

However, at the time of creation of management information for thecompressed and encoded data of the section 2 at STEP 17, the creation isperformed so that the marking position set at the above-described STEP15 is reflected.

Subsequently, a configuration for performing marking according to asubject distance will be described.

Here, a condition that “a time point at which a subject distance fallswithin a range equal to or smaller than a predetermined value is set asa marking timing” is set. In addition, a case where an executiondetermination criterion that “marking is not performed unless one minuteor more has passed from the last marking position to the set markingtiming” is included is cited as an example.

A flowchart of FIG. 6 shows an example of a procedure for executingmarking according to a subject distance.

Firstly, for example, the management information controlling section 20receives information of a subject distance Ls at STEP S301. At thistime, the management information controlling section 20 receivesinformation of the subject distance that is constantly determined by,for example, the camera controlling section 18 for the purpose of theauto focus control. Meanwhile, the information of the subject distanceused here indicates every kinds of control values determined in the autofocus control including an evaluation value obtained on the basis of thecontrast of a video signal in the case of the auto focus control basedon the contrast method, for example.

At STEP 302, a determination of “whether or not a state is shifted froma state where the subject distance Ls is greater than a predeterminedthreshold to a state where the subject distance Ls becomes equal to orsmaller than the threshold” is performed regarding the subject distanceLs received at STEP S301. Suppose that the threshold th is 5m, forexample. Whether or not the state is shifted from a state where thesubject distance Ls is greater than 5m to a state where the subjectdistance Ls becomes equal to or smaller than 5m is determined. If anegative determination result that such a change is not observed isobtained, the processing shown in this drawing is terminated withoutperforming marking. If a positive determination result is obtained, theprocess proceeds to STEP S303.

At STEP S303, the present time Tpre is acquired.

At the next STEP S304, whether or not a predetermined time (one minute)or more has passed from the final (last) marking to the time of thismarking position is determined. If a negative determination result isobtained, the processing is terminated without performing marking. Onthe other hand, if a positive determination result that one minute ormore has passed is obtained, the process proceeds to STEP S305.

At STEP S305, a position on the recording data corresponding to thepresent time Tpre acquired at STEP S303 is set as a sectioning position(a marking position).

In such processing, for example, the threshold th at STEP S302 may beset in the following manner. More specifically, a value corresponding tosubstantially focused subject distance is set as the threshold th. Insuch a case, when a user changes a subject to one existing at a positionfarther than a subject distance corresponding to the threshold th, themarking is performed in response to a state where the subject issubstantially focused on after the auto focus control operates. That is,as a recording result, a position on the image content brought into afocused state from a state where a subject temporarily becomesout-of-focus after the subject has changed is set as a sectioningposition.

As a variation, a configuration for allowing the camera controllingsection 18 to execute part of processing shown in FIG. 6 is alsopossible. FIG. 7 to be described next is based on this configuration.

FIG. 7 is executed by an image capturing apparatus between a start and atermination of recording of one piece of recording data, and is aspecific example of a sequence of processing for recording the recordingdata including processing for marking according to a subject distance.This processing sequence will be described according to numeralsindicating steps shown with ( ) in the drawing. The marking processingin this processing sequence employs the condition and the executiondetermination criterion shown in FIG. 6.

Since STEP 1 to STEP 6 are similar to STEP 1 to STEP 6 of FIG. 3 or FIG.5, the description is omitted here.

In addition, writing of data of a section 1, an update of a file system,and buffering at STEP 7 to STEP 9 are also similar to those of FIG. 3 orFIG. 5.

However, in this drawing, STEP 2-1 and STEP 2-2 are shown as processingof the camera controlling section 18.

At STEP 2-1, processing for determining whether or not the markingcondition is met utilizing information of a subject distance acquiredfor the purpose of the auto focus control is started in response to theuser interface controlling section 22 receiving a recording startoperation at STEP 2. That is, at STEP 2-1 and the following steps, theprocessing corresponding to FIG. 6 is started. The camera controllingsection 18 of this case at least performs processing for acquisition ofthe subject distance Ls and for comparison of the subject distance Lsand the threshold th at STEPs S301 and S302.

Additionally, in this case, regarding the relationship between thesubject distance Ls and the threshold th, Ls>th is maintained during aperiod after the start of STEP 2-1 until STEP 2-2. It is assumed that anegative result is obtained at STEP S302.

In this case, as shown in STEP 2-2, the relationship between the subjectdistance Ls and the threshold th is determined to have shifted from astate of Ls>th to Ls≦th at a timing that is in the course of compressingand encoding image captured/sound captured video/audio data of a section2. That is, a positive determination result is obtained at STEP S302. Inthis case, the camera controlling section 18 notifies the managementinformation controlling section 20 of this.

In this drawing, STEP 10 to STEP 14 are omitted due to thecorrespondence to FIG. 3 and FIG. 5 described above. STEP to bedescribed next is STEP 15.

STEP 15: The management information controlling section 20 notified thatthe subject distance Ls and the threshold th has shifted from the stateof Ls>th to Ls≦th at STEP S2-2 described above executes processing atSTEP S303 and the following step of FIG. 6, for example. In this case, apositive determination result is obtained at STEP S304 and processing ofSTEP S305 is executed since the marking is not performed at the timingbefore STEP 2-2. That is, a position on the recording data correspondingto the present time Tpre is set as a marking position. This markingposition resides within the recording data of the section 2.

Since STEP 16 to STEP 29 are similar to those of FIG. 3 or FIG. 5, thedescription is omitted here. Meanwhile, at the time of creation ofmanagement information for compressed and encoded data of the section 2at STEP 17, the creation is performed so that the marking position setat STEP 15 is reflected as in the case of FIG. 5.

In the description given above, it is assumed that the cameracontrolling section 18 performs the comparison of the subject distanceLs and the threshold th at STEP S302 of FIG. 6. However, as describedbefore, with the configuration of FIG. 1, the management informationcontrolling section 20 may be configured to execute processing of STEPS302 utilizing the subject distance Ls received from the cameracontrolling section 18. In either case, such comparison processing canbe realized as software processing by a CPU of a microcomputer actuallyprovided in the image capturing apparatus.

Subsequently, a configuration for performing marking according to achange in a predetermined feature value regarding an input signal (avideo signal or an audio signal) obtained by image capturing or soundcapturing will be described.

As illustrated above, some features of an input signal can beconsidered. Here, particularly skin colors of colors of images displayedon the basis of a video signal is detected as a feature value. As acondition for detecting skin colors, a condition that “a time point atwhich a ratio (a skin color occupancy ratio α) of skin color detectedblocks in a specific area set as a skin color detection area in an imagearea for one frame has shifted from a value smaller than a thresholdth(h) (α<th(h)) to a value equal to or greater than the threshold th(h)(αth(h)) is set as a marking timing” is set. Satisfying theabove-described condition can be considered as a state where a humanfigure is included in an image area potion serving as the skin colordetection area as a subject. Accordingly, the above-mentioned thresholdth(h) is set in consideration of this.

In addition, an execution determination criterion that “the marking isnot performed unless a set marking timing is after one minute or morefrom the last marking position” is also included.

Image capturing apparatuses that implement such a skin color detectingfunction have been already known. A result of the skin color detectionis used in the camera control or the like. For example, techniques forperforming auto focus control using this area as a target whileconsidering an image area in which skin colors are detected as an areashowing a subject of a human figure have been known.

A flowchart of FIG. 8 shows an example of a procedure for executingmarking according to a result of detecting the skin color occupancyratio as a feature value of an input signal.

The processing shown in this FIG. 8 is realized by the video controllingsection 23 (the video compression processing section 12) and themanagement information controlling section 20 working in cooperation.For example, at least processing of STEP S401 from the processing ofFIG. 8 is processing that is executable by the video controlling section23 sampling video signal data that is undergoing compression andencoding processing in the video compression processing section 12.

At STEP S401, processing for detecting a skin color occupancy ratio α,which is a ratio of areas where skin colors are actually detected in askin color detection area set as a partial area of an image area for oneframe, is executed as the skin color detection processing. A conceptthat serves as an assumption of this processing will be described usingFIG. 10A and FIG. 10B.

FIG. 10A shows an image area for one frame as a matrix of units ofmacroblocks. As is known, the macroblocks are a collective unit of Y,Cr, and Cb constituted by a predetermined number of samples (pixels)formed in the course of compressing and encoding digital video signals,such as the MPEG method. A partial area constituted by a group ofmacroblocks at a predetermined position of the image area that is formedin this manner is set as a skin color detection area ARh. In thisdrawing, the skin color detection area ARh is formed by a group ofsquare macroblocks.

In addition, in this case, the skin color detection area ARh is set tobe located at a substantially center part of the image area for oneframe. Detection of skin colors is to detect a subject of a human figure(particularly, a part of a face) by the color. When a human figure isset as a subject, an image is often captured so that the subject islocated substantially at the center as a composition. Accordingly, theskin color detection area ARh is set at the substantially center part ofthe image area for one frame. The setting of the skin color detectionarea ARh in the image area for one frame described here is only anexample. For example, the skin color detection area ARh may be set at aposition suitably shifted from the center part.

In addition, for example, whether or not a color of a pixel is a skincolor can be determined on the basis of a relationship of each value ofcolor difference signals Cr and Cb among the luminance signal (Y) andthe color difference signals (Cr, Cb). For example, as shown in FIG.10B, coordinate axes of Cr and Cb are considered as the X and Y axes andan area on coordinates constituted by a predetermined range ofcoordinates (Cr, Cb) is conceptually treated as skin colors.

The processing for skin color detection of STEP S401 of FIG. 8corresponds to that shown in a flowchart of FIG. 9, for example.

In this drawing, firstly, a variable m that indicates the number ofmacroblocks treated as skin colors (skin color macroblocks) isinitialized to m=0 at STEP S501.

At the next STEP S502, one detection target macroblock serving as atarget of skin color detection is selected from the skin color detectionarea ARh, and the process processes to at STEP S503 and the followingsteps.

At STEP S503, one detection target pixel is selected from the detectiontarget macroblock. At the next STEP S504, a variable n that indicatesthe number of pixels treated as skin colors (skin color pixels) isinitialized to n=0.

At STEP S505, each value of the color difference signal Cb or Cr of thedetection-target pixel selected at the preceding STEP S503 isrecognized. At the next STEP S506, whether or not coordinates (Cb, Cr)of this recognized color difference signals Cb and Cr reside within theskin color area shown in, for example, FIG. 10B is determined. That is,determination regarding whether or not the detection target pixelselected at STEP S503 is the skin color pixel is performed.

If a negative determination result that the coordinates (Cb, Cr) do notreside within the skin color area, i.e., the detection target pixel isnot the skin color pixel, is obtained at STEP S506, STEPs after STEPS507 to be described later are skipped, and the process proceeds to STEPS510.

On the other hand, if a positive determination result that thecoordinates (Cb, Cr) reside within the skin color area, i.e., thedetection target pixel is the skin color pixel, is obtained at STEPS506, the process proceeds to STEP S507. The variable n indicating thenumber of skin color pixels is incremented to n=n+1, and the processproceeds to STEP S508.

At STEP S508, whether or not the current variable n is equal to orgreater than 2 is determined. In this case, if two or more pixels areskin color pixels among all of pixels forming the macroblock, themacroblock can be considered as a skin color macroblock. By determiningwhether or not the current variable n is equal to or greater than 2 atSTEP S508, whether or not the currently selected check target macroblockis a skin color macroblock is determined.

If a positive determination result is obtained at STEP S508, the processproceeds to STEP S509. At STEP S509, the variable m that indicates thenumber of skin color macroblocks is incremented to m=m+1, and theprocess proceeds to STEP S511.

On the other hand, if a negative determination result is obtained atSTEP S508, the process proceeds to STEP S510.

At STEP S510, whether or not the currently selected detection targetpixel is the last selected detection target pixel in the currentlyselected detection target macroblock is determined. If a negativedetermination result is obtained here, the process returns to STEP S503to perform skin color detection regarding the next detection targetpixel.

On the contrary, if a positive result is obtained at STEP S510, theprocess proceeds to STEP S511.

At STEP S511, whether or not the currently selected detection targetmacroblock is the last selected macroblock among all of macroblocks inthe skin color detection area ARh is determined. If a negativedetermination result is obtained here, the process returns to STEP S502to perform skin color detection regarding the rest of the macroblocks.On the contrary, if a positive determination result is obtained at STEPS511, the process proceeds to STEP S512.

At the stage of STEP S512, the number of macroblocks detected as skincolor macroblocks (the number of skin color macroblocks m) isestablished as a result of performing the skin color detection regardingall of macroblocks in the skin color detection area ARh.

Accordingly, at STEP S512, the skin color occupancy ratio α isdetermined utilizing the number of skin color macroblocks m. As thesimplest example, the skin color occupancy ratio α can be calculatedthrough a calculation of α=m/M, where the number of all of macroblocksforming the skin color detection area ARh is M.

The description returns to FIG. 8.

After detecting the skin color occupancy ratio α through the procedureof the above-described FIG. 9 as STEP S401, the process proceeds to STEPS402.

The processing of STEP S402 may be executed by the video controllingsection 23 or may be executed by the management information controllingsection 20. However, the description is given here while assuming thevideo controlling section 23 executes the processing. The videocontrolling section 23 determines whether or not a state has shiftedfrom a state where the skin color occupancy ratio α is lower than thethreshold th(h) (α<th(h)) to a state where the skin color occupancyratio α is equal to or higher than the threshold th(h) (αth(h)) as aresult of comparing the skin color occupancy ratio α detected at theabove-described STEP S401 and a predetermined threshold th. As describedabove, a fact that the state has shifted from the state where the skincolor occupancy ratio α is lower than the threshold th(h) to the statewhere the skin color occupancy ratio α is equal to or higher than thethreshold th(h) means that a change, which is that a subject of a humanfigure is included in the image area corresponding to the skin colordetection area ARh, has occurred.

If a negative determination result is obtained at STEP S402, theprocessing shown in this drawing is terminated. If a positivedetermination result is obtained, the process proceeds to STEP S403.

The processing at STEP S403 and the following steps is processingexecuted by the management information controlling section 20 inresponse to the obtainment of the positive determination result at STEPS402. At STEP S403, the present time Tpre is acquired. At the next STEPS404, whether or not a predetermined time (one minute) or more haspassed from the final (last) marking position to the time for thismarking position is determined. If a negative determination result isobtained, this processing is terminated. If a positive determinationresult is obtained, the process proceeds to STEP S405. A position on therecording data corresponding to the present time Tpre acquired at STEPS303 is set as a sectioning position (a marking position).

FIG. 11 is executed by an image capturing apparatus between a start anda termination of the recording of one piece of recording data and is aspecific example of a sequence of processing for recording the recordingdata including processing for marking according to the skin colordetection result. This processing sequence will be described accordingto numerals indicating steps shown with ( ) in the drawing. The markingprocessing in this processing sequence employs the condition and theexecution determination criterion shown in FIG. 8.

Since STEP 1 to STEP 6 are similar to STEP 1 to STEP 6 of FIG. 3, FIG.5, or FIG. 7, the description is omitted here.

In addition, writing of data of a section 1, an update of a file system,and buffering at STEP 7 to STEP 9 are also similar to those of FIG. 3,FIG. 5, or FIG. 7.

However, in this drawing, STEP 3-1 and STEP 3-2 are shown as processingof the video compression processing section 12.

At STEP 3-1, processing for comparing the skin color occupancy ratio αand the threshold th(h) is started in response to the user interfacecontrolling section 22 receiving a recording start operation at STEP 2.That is, execution of processing shown at STEPs S401 and S402 of FIG. 8is started.

Additionally, in this case, regarding the relationship between the skincolor occupancy ratio α and the threshold th, α<th(h) is maintainedduring a period after the start of STEP 3-1 until STEP 3-2. It isassumed that a negative result is obtained at STEP S402.

In this case, as shown in STEP 3-2, the relationship between the skincolor occupancy ratio α and the threshold th(h) is determined to haveshifted from the state of α<th(h) to α th(h) at a timing that is in thecourse of compressing and encoding image captured/sound capturedvideo/audio data of a section 2. That is, a positive determinationresult is obtained at STEP S302. The video controlling section 23 forcontrolling video compression processing notifies the managementinformation controlling section 20 of this.

In this drawing, STEP 10 to STEP 14 are omitted due to thecorrespondence to FIG. 3 and FIG. 5 described above as in the case ofFIG. 7. STEP to be described next is STEP 15.

STEP 15: The management information controlling section 20 notified atSTEP 3-2 described above executes processing at STEP S403 and thefollowing step of FIG. 8, for example. In this case, a positivedetermination result is obtained at STEP S404 and processing of STEPS405 is executed since the marking is not performed at the timing beforeSTEP 3-2. That is, a position on the recording data corresponding to thepresent time Tpre is set as a marking position. This marking positionalso resides within the recording data of the section 2.

Since STEP 16 to STEP 29 are similar to those of FIG. 3, FIG. 5, or FIG.7, the description is omitted here. Meanwhile, at the time of creationof management information corresponding to the compressed and encodeddata of the section 2 at STEP 17, the creation is performed so that themarking position set at STEP 15 is reflected as in the case of FIG. 5and FIG. 7.

The management information controlling section 20 may be configured toexecute processing for comparing the skin color occupancy ratio α andthe threshold th(h) at STEP S402 of FIG. 8 after receiving the skincolor occupancy ratio α from the camera controlling section 18.

Subsequently, a configuration for performing marking according to achange in a position of an image capturing apparatus will be described.

In this case, a condition that “a time point a which a predeterminedtime has passed from a time point at which a distance between apredetermined reference position and the present position becomes equalto and greater than a predetermined value is set as a marking timing” isset. In addition, an execution determination criterion that “marking isnot performed unless one minute or more has passed from the last markingposition to the set marking timing” is included.

A flowchart of FIG. 12 shows an example of a procedure for executingmarking according to a change in position of an image capturingapparatus.

The management information controlling section 20 is configured to startprocessing from STEP S601 at predetermined intervals. As processing ofSTEP S601, present position information pos_p detected by the positiondetecting section 24 is received through the camera controlling section18. That is, the reception of the present position information pos_p isperformed at predetermined intervals. At STEP S602, a distance Disbetween a reference position and the present position is calculatedusing the present position information pos_p received at theabove-described STEP S601 and reference position information pos_rindicating the reference position.

Meanwhile, for example, setting of a position for a time of a start ofrecording as the reference position is possible. The reference positioninformation pos_r of this case can be obtained on the basis of theposition information detected at the time of the start of the recording.

Alternatively, a user may specify the reference position beforehand. Forexample, in response to a user, carrying the image capturing apparatus,performing an operation for setting the reference position, the positioninformation detected at that time is set as the reference positioninformation pos_r. In addition, the image capturing apparatus isconfigured to output map information and a configuration for setting alocation specified on this map information, a location corresponding aname of a place or a name of a station as the reference positioninformation pos_r is also possible.

At STEP S603, whether or not a state has shifted from a state ofDis<th(D) to a state of Dis th(D) is determined regarding the distanceDis between the reference position and the present position calculatedat the above-described STEP S602 and a threshold th(D) corresponding toa previously set predetermined distance. That is, “whether or not thedistance between the predetermined reference position and the presentposition has shifted from the state where the distance is smaller thanthe predetermined value to the state where the distance is equal to orgreater than the predetermined value” is determined. If a negativedetermination result that such a change is not observed is obtained atSTEP S603, the processing shown in this drawing is terminated withoutperforming the marking. If a positive determination result is obtained,the process proceeds to STEP S604.

At STEP S604, the present time Tpre is acquired.

At the next STEP S605, whether or not a predetermined time (one minute)or more has passed since the final (last) marking to the time for thismarking position is determined. If a negative determination result isobtained, the processing is terminated without performing the marking.On the contrary, if a positive determination result that one minute ormore has passed is obtained, the process proceeds to STEP S606.

At STEP S606, a position on the recording data corresponding to Tpre−1second is set as a sectioning position (a marking position).

Meanwhile, a configuration for allowing the camera controlling section18 to perform part of processing shown in the above-described FIG. 11 isalso possible. FIG. 13 to be described next is based on thisconfiguration.

FIG. 13 is executed by an image capturing apparatus between a start anda termination of recording of one piece of recording data and is aspecific example of a sequence of processing for recording the recordingdata including processing for marking according to a position change ofthe image capturing apparatus. This processing sequence will bedescribed according to numerals indicating steps shown with ( ) in thedrawing. The marking processing in this processing sequence employs thecondition and the execution determination criterion shown in FIG. 11.

Since STEP 1 to STEP 6 are similar to STEP 1 to STEP 6 of FIG. 3, FIG.5, or FIG. 7, the description is omitted here.

In addition, writing of data of a section 1, an update of a file system,and buffering at STEP 7 to STEP 9 are also similar to those of FIG. 3,FIG. 5, or FIG. 7.

However, in this drawing, STEP 2-1 and STEP 2-2 are shown as processingof the camera controlling section 18.

At STEP 2-1, processing for determining whether or not the markingcondition are met utilizing the present position information receivedfrom the position detecting section 24 at predetermined intervals isstarted in response to the user interface controlling section 22receiving a recording start operation at STEP 2. That is, at STEP 2-1and the following steps, the processing corresponding to FIG. 12 isstarted. The camera controlling section 18 of this case at leastperforms processing for acquisition of the present position informationpos_p, for calculation of the distance Dis, and for comparison of thepresent position information pos_p and the distance Dis at STEPs S601 toS603, for example.

Additionally, in this case, regarding the relationship between thepresent position information pos_p and the distance Dis, Dis<th(D) ismaintained during a period after the start of STEP 2-1 until STEP 2-2.It is assumed that a negative result is obtained at STEP S603.

In this case, as shown in STEP 2-2, the relationship between the presentposition information pos_p and the distance Dis is determined to haveshifted from the state of Dis<th(D) to a state of Dis th(D) at a timingthat is in the course of compressing and encoding image captured/soundcaptured video/audio data of a section 2. That is, a positivedetermination result is obtained at STEP S603. In this case, the cameracontrolling section 18 notifies the management information controllingsection 20 of this.

In this drawing, STEP 10 to STEP 14 are omitted due to thecorrespondence to FIG. 3 and FIG. 5 as in the case of FIG. 7. STEP to bedescribed next is STEP 15.

STEP 15: The management information controlling section 20 notified thatthe relationship between the present position information Pos_p and thedistance Dis has shifted from the state of Dis<th(D) to Dis th(D) atSTEP 2-2 described above executes processing at STEP S604 and thefollowing step of FIG. 12. In this case, a positive determination resultis obtained at STEP S605 and processing of STEP S606 is executed sincethe marking is not performed at the timing before STEP 2-2. That is, aposition on the recording data corresponding to the present time Tpre isset as a marking position. This marking position resides within therecording data of the section 2.

Since STEP 16 to STEP 29 are similar to those of FIG. 3, FIG. 5, or FIG.7, the description is omitted here. Meanwhile, at the time of creationof management information corresponding to the compressed and encodeddata of the section 2 at STEP 17, the creation is performed so that themarking position set at STEP 15 is reflected as in the case of FIG. 5.

Also in this case, it is assumed that the camera controlling section 18performs comparison of the present position information pos_p and thedistance Dis at STEP S603 of FIG. 12 in the description given above.However, as described above, with the configuration of FIG. 1, themanagement information controlling section 20 may configured to executethe processing of STEP S603 utilizing the present position informationpos_P received from the camera controlling section 18.

Example manners of storing information of marking positions set in theabove-described manner in the management information will be cited here.

Here, a case where the recording medium is a DVD (Digital VersatileDisc) is cited as an example. As is known, DVD-VR (Video Recording) andDVD-Video are widely known as logical data formats of DVD.

In the case of DVD-VR,

it is possible to store information of marking positions utilizing astructure of

RTR_VMG→UD_PGCI→CI→M_CI→M_C_EPI. In this case, according to the DVD-VRformat, the information of the marking positions can be set at anaccuracy of frame. A time for one frame is 1/30 second in the NTSCsystem and 1/25 second in the PAL system.

Additionally, in the case of DVD-Video,

the information of marking positions can be stored utilizing a structureof

VTSI→VTS_PGCIT→VTS_PGCI→C_PBIT→C_PBI.

Subsequently, utilization examples of marking positions set regardingrecording data in the above-described manner will be described.

As shown in FIG. 14, suppose that three titles are recorded as examplesof recorded contents of recording data recorded on, for example, thestorage section 17. Here, the recording data of a unit of title has thecontent from a recording start operation to a recording stop operation.Firstly, each title is managed so that a recording start position and arecording end position are specified by management information.

On that basis, in this case, one marking position is set between a startand a termination of the recording regarding recording data of a title1. Accordingly, recording data of the title 1 can be managed whileassuming that the recording data is divided into a recording dataportion from the recording start position to a data position immediatelybefore the marking position and a recording data portion from themarking position to the recording end position by this one markingposition. The recording data portions divided in this manner is handledas marking chanters, and serial numbers from 1 are sequentially attachedfrom the recording start position. Accordingly, the recording data ofthe title 1 can be considered to be constituted by marking chapters 1and 2. In addition, since three marking positions are set regardingrecording data of a title 2, the recording data of the title 2 aredivided into four marking chapters of marking chapters 1, 2, 3, and 4.Additionally, since two marking positions are set regarding recordingdata of a title 3, the recording data of the title 3 is divided intothree marking chapters of marking chapters 1, 2, and 3.

For the confirmation, the information of marking positions of each titleis also stored in management information as in the case of the recordingstart position and the recording end position in this embodiment.

In this embodiment, utilization of marking chapters of each tile, whichis divided by marking positions in this manner, in the following manneris possible.

One is to allow forwarding and rewinding of the recording data in a unitof this marking chapter regarding a jump to a playback position duringplayback.

Another is to allow a content of recording data recorded on a recordingmedium to be presented in a unit of this marking chapter when thecontent is presented through displaying.

FIG. 15A to FIG. 15C show display examples of index screens representingthe content recorded on a recording medium through displaying.Meanwhile, the display of the index screens shown in this drawingcorresponds to the recorded content shown in FIG. 14.

Firstly, FIG. 15A shows a display example of a basic index screencorresponding to the recorded content shown in FIG. 14. As this basicindex display, thumbnail images corresponding to titles are arranged anddisplayed in an order of playback as shown in the drawing. Since threetitles of the titles 1, 2, and 3 are recorded as the recorded content ofFIG. 14, a thumbnail image of each title is arranged and displayed in anorder of the title 1→the title 2→the title 3 in FIG. 15A.

Suppose that an operation for selecting the title 2 to expand the titleto chapters is performed in this state. In response to this, the displayof the recorded content changes from the manner shown in FIG. 15A to themanner shown in FIG. 15B.

More specifically, in FIG. 15B, thumbnail display is performed in a unitof marking chapters regarding the title 2. As a whole, thumbnail imagesare displayed in an order of the title 1→the title 2>the marking chapter1→the title 2>the marking chapter 2→the title 2>the marking chapter3→the title 2>the marking chapter 4→the title 3.

FIG. 15C shows a display state where thumbnail display is arranged in aunit of marking chapters also regarding the title 1 and the title 3 in amanner similar to the above-described one. In this case, as a whole,thumbnail images are displayed in an order of the title 1>the markingchapter 1→the title 1>the marking chapter 2→the title 2>the markingchapter 1→the title 2>the marking chapter 2→the title 2>the markingchapter 3→the title 2>the marking chapter 4→the title 3>the markingchapter 1→the title 3>the marking chapter 2→the title 3>the markingchapter 3.

For example, a configuration for performing index display only in a unitof marking chapter without employing a hierarchical concept of a title>amarking chapter such as this embodiment is also possible (In this case,the display shown in FIG. 15C is the basis display). However, since thedistinction between a start position of a title and marking positionsbecomes clear by displaying the recorded content so that theabove-described hierarchical concept is reflected as shown in FIG. 15Ato FIG. 15C, the index screen is easily viewable for users and retrievalof recording data becomes easier.

Depending on an operation for such index display, it is possible tostart playback of recording data not only from the top of a title butalso from the top of a chapter. If a predetermined operation forstarting playback by specifying, for example, one of the markingchapters of the title 2 shown in FIG. 15B or a desired thumbnail imagefrom the marking chapters of each title shown in FIG. 15C, the apparatusoperates to start playback of the recording data from the position ofthe marking chapter corresponding to the specified thumbnail image.Meanwhile, in the case of a state where thumbnail images of only titlesare displayed as shown in FIG. 15A or in the case of performing anoperation for starting playback by specifying a thumbnail image that isnot expanded to the chapter level such as the title 1 and the title 3 inFIG. 15B, the playback of the recording data is started from the topposition of the title.

FIG. 16 shows a configuration of an image capturing apparatus of thisembodiment supporting a function for playing back data recorded (stored)on the storage section 17. In this drawing, the same numerals areattached to sections that can be treated the same as those of FIG. 1.

Recording data and management information are recorded on the storagesection 17 (a recording medium) of this case. In addition, themanagement information controlling section 20 of this case reads out themanagement information from the storage section 17 and holds theinformation therein, and is shown as a part for executing control ofplayback regarding the recording medium of the storage section 17. Sincethe management information controlling section 20 executes the controlof reading out of data that is targeted at the storage section 17 at,for example, a logical level of a file system, the image capturingapparatus includes an access controlling section 31 between themanagement information controlling section 20 and the storage section 17in order to realize physical access to the storage section 17 (arecording medium).

The user interface controlling section 22 of this case is shown as apart for transferring instructions and commands, corresponding tooperations regarding execution of displaying of recorded content asshown in FIG. 15A to FIG. 15C and operations regarding playback ofrecording data, to the management information controlling section 20.The management information controlling section 20 executes access to thestorage section 17 if necessary according to the instructions from theuser interface controlling section 22.

The recording data read out from the storage section 17 according to theaccess control of the management information controlling section 20 isinput to a video/audio demultiplexing processing section 32 at the timeof typical playback. Video data and audio data recorded on the storagesection 17 as the recording data is in a format that is multiplexedafter having undergone compression and encoding. The video/audiodemultiplexing processing section 32 demultiplexes the input recordingdata into the video data and the audio data, and outputs the video datato a video signal processing section 35 and the audio data to an audiosignal processing section 33.

Compression and encoding processing has been performed on the video dataand the audio data input to the video signal processing section 35 andthe audio signal processing section 33. Accordingly, the video signalprocessing section 35 executes decoding processing for decompression onthe input video data. In addition, the video signal processing section35 performs predetermined signal on the decoded video data if necessary.Similarly, the audio signal processing section 33 executes decodingprocessing for decompression on the input audio data. The audio signalprocessing section 33 performs predetermined signal processing on thedecoded audio data if necessary. Since the video data output from thevideo signal processing section 35 and the video data output from theaudio signal processing section 33 are in a state where playback timesare synchronized and so-called lip-sync is maintained.

The video data output from the video signal processing section 35 isoutput to a display processing section 36. The display processingsection 36 executes control processing for the display driving on adisplay section 37 based on the input video data. This causes playedback images of the video data to be displayed on the display section 37.

The audio data output from the audio signal processing section 33 isinput to an audio output section 34. The audio output section 34performs signal processing and amplification for outputting the inputaudio data from, for example, a speaker or the like as audio. In thismanner, playback of images and audio of the recording data read out fromthe storage section 17 is performed. In addition, in images and audiooutput in this manner, the above-described lip-sync is maintained. Thisdrawing shows a configuration for playing back and outputting images andaudio of the recording data by a single image capturing apparatus.However, for example, analog or digital video signals and audio signalsmay be output to the outside from an image capturing apparatus and anexternal display apparatus and an audio output apparatus may output theimages and audio.

In addition, an index display processing section 38 is provided forperforming index display shown in, for example, FIG. 15A to FIG. 15C.

For example, if an operation for executing the index display isperformed, the user interface controlling section notifies themanagement information controlling section of this. The managementinformation controlling section recognizes the recorded content storedon the storage section 17 with reference to the management informationhaving been read out from the storage section 17 and held therein. Onthe basis of this recognition result, the management informationcontrolling section 20 reads out information necessary for the indexdisplaying from the storage section 17 and transfers the information tothe index display processing section 38. For example, image data that issources of thumbnail images to be mapping-displayed on an index screenis read out from the recording data of the storage section 17, and theimage data is transferred to the index display processing section 38. Inthe case of a logical format that thumbnail images are previouslygenerated for each of starts of the titles and starts of the chaptersand are recorded together with the recording data, the managementinformation controlling section 20 has only to read out necessarythumbnail image data from the storage section 17 and transfer the datato the index display processing section 38.

The index display processing section 38 then generates image data of anindex screen shown in FIG. 15A to FIG. 15C utilizing the transferredimage data, and transfers the generated image data to the displayprocessing section 36. This causes the index display to be performed onthe display section 37.

For example, a display mode of showing recorded content in a unit oftitles or marking chapters in a simpler manner only by using textinformation is also possible as the index display by the index displayprocessing section 38.

Meanwhile, the present invention is not limited to the embodimentshaving been described.

Firstly, two or more states may be targeted and combined as states of animage capturing apparatus for setting marking positions. In thedescription given above, a zoom operation, a state of movement of animage capturing apparatus itself, a subject distance, and a featurevalue of an input signal are cited as the states of the image capturingapparatus. However, for example, a configuration of performing markingif one of the condition that the zoom operation is terminated and thecondition that the subject distance becomes equal to or smaller than apredetermined value is met is also possible. In addition, aconfiguration for setting a marking execution determination criterion byemploying the logical multiplication of conditions, such as the markingis performed if both of the condition that the zoom operation isterminated and the condition that the subject distance becomes equal toor smaller than a predetermined value are met, is also possible.

Regarding kinds of actual recording media (storage media) compliant withthe storage section 17, optical disc recording media other than a DVDand an HDD that have been cited as examples and magnetic disc recordingmedia can also be adopted. In addition, semiconductor storage devices,such as flash memories, can be also employed. In addition, the recordingmedia, such as magnetic tapes, can be employed.

Additionally, in the embodiments, it is assumed that informationindicating marking positions is stored in a logical area different fromthat for recording data and is included in a structure of managementinformation for managing the recording data. However, the informationmay be recorded using a structure other than this. For example,regarding the marking position information, image data and audio datathat can present a marking position to users may be superimposed on avideo signal and an audio signal forming the recording data.

In addition, an image capturing apparatus according to the presentinvention is not limited to, for example, a portable video camera, andis applied to every kind of electronic devices having an image capturingfunction, such as, for example, mobile phones.

1. An image capturing apparatus comprising: captured image acquiringmeans for acquiring image information through image capturing; recordingmeans for recording, as recording information, at least the imageinformation acquired by the captured image acquiring means as movingimage information; apparatus state detecting means for detecting thatthe image capturing apparatus is in a predetermined state while imagecapturing for acquiring the moving image information to be recorded bythe recording means is being performed; and sectioning position settingmeans for setting a sectioning position on the moving image informationon the basis of a timing at which the apparatus state detecting meanshas detected the predetermined state.
 2. The image capturing apparatusaccording to claim 1, wherein the captured image acquiring meansincludes a view angle changing means for changing an angle of view of anoptical system for image capturing in response to an operation, andwherein the apparatus status detecting means is configured to detect, asthe predetermined state of the image capturing apparatus, occurrence ofa predetermined state transition regarding an operation for changing theangle of view by the view angle changing means.
 3. The image capturingapparatus according to claim 1, comprising: apparatus movement detectingmeans for detecting movement of the image capturing apparatus itself,wherein the apparatus state detecting means is configured to detect, asthe predetermined state of the image capturing apparatus, occurrence ofa predetermined state transition regarding the movement detected by theapparatus movement detecting means.
 4. The image capturing apparatusaccording to claim 1, wherein the captured image acquiring means isconfigured to generate subject distance information for indicating adistance to a subject that is utilized in auto focus control, andwherein the apparatus state detecting means is configured to detect, asthe predetermined state of the image capturing apparatus, occurrence ofa predetermined change regarding the subject distance informationgenerated by the captured image acquiring means.
 5. The image capturingapparatus according to claim 1, wherein the apparatus state detectingmeans is configured to detect, as the predetermined state of the imagecapturing apparatus, occurrence of a predetermined change in apredetermined feature value in the image information acquired by thecaptured image acquiring means.
 6. The image capturing apparatusaccording to claim 1, comprising: audio acquiring means for acquiringaudio information, which is information that may be recorded by therecording means together with the image information, by capturingexternal audio, wherein the apparatus state detecting means isconfigured to detect, as the predetermined state of the image capturingapparatus, occurrence of a predetermined change in a predeterminedfeature value in the audio information.
 7. The image capturing apparatusaccording to claim 1, comprising: position detecting means for detectingpresent position information indicating a present position of theapparatus, wherein the apparatus state detecting means is configured todetect, as the predetermined state of the image capturing apparatus,occurrence of a predetermined change in a relationship between thepresent position information detected by the position detecting meansand predetermined position information other than the present positioninformation.
 8. The image capturing apparatus according to claim 7,wherein the apparatus state detecting means is configured to detect, asthe predetermined state of the image capturing apparatus, occurrence ofa change that a distance between positions indicated by thepredetermined position information and the present position informationbecomes equal to or greater than or becomes equal to or smaller than apredetermined value.
 9. The image capturing apparatus according to claim7, wherein the predetermined position information is present positioninformation detected by the position detecting means at the timing thatthe sectioning position setting means has set a sectioning position lasttime.
 10. The image capturing apparatus according to claim 7, whereinthe predetermined position information is position informationindicating a position specified through a specifying operation.
 11. Theimage capturing apparatus according to claim 1, wherein the sectioningposition setting means is configured to set, as the sectioning position,a position on the moving image information corresponding to a time pointat which a predetermined time has passed from a time point at which theapparatus state detecting means has acquired a predetermined detectionresult.
 12. The image capturing apparatus according to claim 1, whereinthe sectioning position setting means is configured to set, as thesectioning position, a position on the moving image informationcorresponding to a time point that is a predetermined time before a timepoint at which the apparatus state detecting means has acquired apredetermined detection result.
 13. The image capturing apparatusaccording to claim 1, comprising: recorded content display controllingmeans for controlling, on the basis of setting of the sectioningposition regarding the recorded moving image information, displayshowing a content of the recorded moving image information so that thedisplay is performed in a predetermined manner where the sectioningposition is reflected.
 14. The image capturing apparatus according toclaim 1, comprising: playback means capable of performing playback usingthe sectioning position having been set on the recorded moving imageinformation as a playback start position when playing back the recordedmoving image information.
 15. A recording method comprising: a capturedimage acquiring step of acquiring image information through imagecapturing; a recording step of recording, as recording information, atleast the image information acquired at the captured image acquiringstep as moving image information; an apparatus state detecting step ofdetecting that the image capturing apparatus is in a predetermined statewhile image capturing for acquiring the moving image information to berecorded at the recording step is being performed; and a sectioningposition setting step of setting a sectioning position on the movingimage information on the basis of a timing at which the predeterminedstate having been detected at the apparatus state detecting step.